The invention relates to a method and a computer program with program code for determining a future system response of a dynamic system.
It is known from S. Haykin, “Neural Networks: A Comprehensive Foundation,” Prentice Hall, Second Edition, ISBN 0-13-273350-1, pages 732-789, 1999 (“the Haykin reference”) that a neural structure, for example a neural network, can be used to describe and model a dynamic system or dynamic process and its process response.
Generally a dynamic system or dynamic process is described by a status transition description that is not visible to an observer of the dynamic process and an output equation that describes observable quantities of the technical dynamic process.
Such a process response of a dynamic process is shown in FIG. 2.
The dynamic process 200 or dynamic system 200, in which the dynamic process operates, is subject to the influence of an external input quantity u of predefinable dimension, with an input quantity ut at a point in time t being referred to as:ut∉1,where 1 refers to a natural number.
The input quantity at a point in time t causes a change in the dynamic process.
An internal status st (stεm) of predefinable dimension m at a point in time t cannot be observed by an observer of the dynamic system 200.
A status transition of the internal status st of the dynamic process is caused as a function of the internal status st and the input quantity ut and the status of the dynamic process switches to a successor status st+1 at a subsequent point in time t+1.
The following is thereby valid:St+1=f(st,ut).  (1)where f(.) refers to a general mapping rule.
An output quantity yt that can be observed by an observer of the dynamic system 200 at a point in time t is a function of the input quantity ut and the internal status st.
The output quantity yt (ytεn) is a predefinable dimension n.
The dependency of the output quantity yt on the input quantity ut and the internal status st of the dynamic process is defined by the following general rule:yt=g(st),  (2)where g. refers to a general mapping rule.
To describe the dynamic system 200 in the Haykin reference a neural structure is used comprising computer elements connected to one another in the form of a neural network of neurons connected to one another. The connections between the neurons of the neural network are weighted. The weightings of the neural network are combined in a parameter vector v.
An internal status of a dynamic system that underlies a dynamic process is thus a function, according to the following rule, of the input quantity ut and the internal status of the preceding point in time st and the parameter vector v:st+1=NN(v,st,ut)  (3)where NN(.) refers to a mapping rule predefined by the neural network.
This description of the dynamic system 200 according to equation (3) is also referred to as the forecast approach.
Alternatively the dynamic system can also be described by:st=f(st−1,ut)  (1′)withst=NN(v,st−1,ut)  (3′)which is referred to as the consistency approach. The forecast approach and consistency approach result in minor structural differences in the respective network structures but are equivalent, alternative forms of description for dynamic systems.
A further neural structure to describe the dynamic system 200, a neural network referred to as a time delay recurrent neural network (TDRNN), is known from David E. Rumelhart et al., “Parallel Distributed Processing, Explorations in the Microstructure of Cognition”, vol. 1: Foundations, A Bradford Book, The MIT Press, Cambridge, Mass., London, England, 1987 (“David E. Rumelhart et al.”).
The known TDRNN is shown in FIG. 5 as a neural network 500 developed over a finite number of points in time (5 points in time shown: t−4, t−3, t−2, t−1, t).
The neural network 500 shown in FIG. 5 has an input layer 501 having five sub-input layers 521, 522, 523, 524 and 525 that respectively contain a predefinable number of input computer elements, to which input quantities ut−4, ut−3, ut−2, ut−1, ut can be applied at predefinable points in time t−4, t−3, t−2, t−1, t, i.e. time row values with predefined time steps described below.
Input computer elements, i.e. input neurons, are connected via variable connections to neurons of a predefinable number of hidden layers 505 (5 hidden layers shown).
Neurons of a first 531, a second 532, a third 533, a fourth 534 and a fifth 535 hidden layer are thereby connected respectively to neurons of the first 521, second 522, third 523, fourth 524 and fifth 525 sub-input layer.
The connections between the first 531, second 532, third 533, fourth 534 and fifth 535 hidden layer and the first 521, second 522, third 523, fourth 524 and fifth 525 sub-input layer respectively are the same in each instance. The weightings of all connections are respectively contained in a first connection matrix B1.
The outputs of neurons of the first hidden layer 531 are also connected to inputs of neurons of the second hidden layer 532 according to a structure defined by a second connection matrix A1. The outputs of neurons of the second hidden layer 532 are connected to inputs of neurons of the third hidden layer 533 according to a structure defined by the second connection matrix A1. The outputs of neurons of the third hidden layer 533 are connected to inputs of neurons of the fourth hidden layer 534 according to a structure defined by the second connection matrix A1. The outputs of neurons of the fourth hidden layer 534 are connected to inputs of neurons of the fifth hidden layer 535 according to a structure defined by the second connection matrix A1.
In the hidden layers, the first hidden layer 531, the second hidden layer 532, the third hidden layer 533, the fourth hidden layer 534 and the fifth hidden layer 535, “internal” statuses or “internal” system statuses st−4, st−3, st−2, st−1, and st of a dynamic process described by the TDRNN are represented respectively at five successive points in time t−4, t−3, t−2, t−1 and t.
The particulars in the indices in the respective layers respectively indicate the point in time t−4, t−3, t−2, t−1 and t, to which the signals that can be taken or supplied respectively at the outputs of the respective layer refer (ut−4, ut−3, ut−2, ut−1, ut).
An output layer 520 comprises five sub-output layers, a first sub-output layer 541, a second sub-output layer 542, a third sub-output layer 543, a fourth sub-output layer 544 and a fifth sub-output layer 545. Neurons of the first sub-output layer 541 are connected to neurons of the first hidden layer 531 according to a structure defined by an output connection matrix C1. Neurons of the second sub-output layer 542 are connected to neurons of the second hidden layer 532 also according to the structure defined by the output connection matrix C1. Neurons of the third sub-output layer 543 are connected to neurons of the third hidden layer 533 according to the output connection matrix C1. Neurons of the fourth sub-output layer 544 are connected to neurons of the fourth hidden layer 534 according to the output connection matrix C1. Neurons of the fifth sub-output layer 545 are connected to neurons of the fifth hidden layer 535 according to the output connection matrix C1. The output quantities for a respective point in time t−4, t−3, t−2, t−1, t can be taken at the neurons of the sub-output layers 541, 542, 543, 544 and 545 (yt−4, yt−3, yt−2, yt−1, yt)
The principle that equivalent connection matrices in a neural network exhibit the same values at a respective point in time is referred to as the so-called shared weights principle.
The system known from David E. Rumelhart et al. and referred to as a time delay recurrent neural network (TDRNN) is trained in a training phase such that a respective target quantity ytd at a real dynamic system is determined for an input quantity ut. The tupel (input quantity, determined target quantity) is referred to as training datum. A plurality of such training data forms a training data set.
Temporally successive tupels (ut−4, yt−4d), (ut−3, yt−3d), (ut−2, yt−2d) of the points in time (t−4, t−3, t−3, . . . ) of the training data set each exhibit a predefined time step.
The TDRNN is trained with the training data set. An overview of various training methods can likewise be found in the Haykin reference and WO00/08599.
It should be emphasized at this point that only the output quantities yt−4, yt−3, . . . , yt at points in time t−4, t−3, . . . , t of the dynamic system 200 can be identified. The “internal” system statuses st−4, st−3, . . . , st cannot be observed.
The following cost function E is usually minimized in the training phase:
                              E          =                                                    1                T                            ⁢                                                ∑                                      t                    =                    1                                    T                                ⁢                                                      (                                                                  y                        t                                            -                                              y                        t                        d                                                              )                                    2                                                      ->                          min                              f                ,                g                                                    ,                            (        4        )            where T refers to a plurality of points in time taken into consideration.
Developments of the neural structure known from David E. Rumelhart et al. and referred to as a time delay recurrent neural network (TDRNN) are known from WO00/55809 and Zimmermann H. G., Neunier R., Grothmann R., “Modelling of Dynamic Systems by Error-Correction-Neural-Networks”, in Soofe and Cao (eds.), Forecasting Financial Data, Kluwer Verlag, ISBN 0792376803, 2002 (“Zimmermann H. G. et al.”).
The developments from WO00/55809 are particularly suitable for determining future statuses of a dynamic process, which is referred to as “overshooting”.
FIG. 1a from WO00/55809 shows a basic structure, on which the developments known from WO00/55809 are based.
The basic structure is a neural network developed over three points in time t, t+1, t+2.
It has an input layer, having a predefinable plurality of input neurons, to which input quantities ut can be applied at predefinable points in time t, i.e. time row values with predefined time steps as described below.
The input neurons are connected via variable connections to neurons of a predfinable plurality of hidden layers (3 hidden layers shown).
Neurons of a first hidden layer are thereby connected to neurons of the first input layer.
The connection between the first hidden layer and the first input layer has weightings, which are contained in a first connection matrix B.
The outputs of neurons of the first hidden layer are also connected to inputs of neurons of a second hidden layer according to a structure defined by a second connection matrix A. The outputs of neurons of the second hidden layer are connected to inputs of neurons of a third hidden layer according to a structure defined by the second connection matrix A.
In the hidden layers, the first hidden layer, the second hidden layer and the third hidden layer, “internal” statuses or “internal” system statuses st, st+1, st+2 of the described dynamic process are represented respectively at three successive points in time t, t+1, t+2.
The particulars in the indices in the respective layers respectively indicate the point in time t, t+1, t+2, to which the signals (ut) that can be taken or supplied respectively at the outputs of the respective layer refer.
An output layer 120 has two sub-output layers, a first sub-output layer and a second sub-output layer. Neurons of the first sub-output layer are connected to neurons of the first hidden layer according to a structure defined by an output connection matrix C. Neurons of the second sub-output layer are connected to neurons of the second hidden layer also according to the structure defined by the output connection matrix C.
The output quantities for a respective point in time t+1, t+2 can be taken at the neurons of the sub-output layers (yt+1, yt+2).
A development of this basic structure from WO00/55809 is shown in FIG. 6.
Developments of the TDRNN structure from Zimmermann H. G. et al., so-called error correction recurrent neural networks (ECRNN), relate to a structurally necessary error correction mechanism, which is integrated as a structural component in a neural structure. FIG. 7 shows a basic structure with corresponding functional equations for an ECRNN.
A further neural structure to describe the dynamic system 200, a neural network referred to as a causal-retro-causal time delay recurrent neural network or causal-retro-causal neural network (CRCNN) is known from WO02/27654.
FIG. 3 shows a basic structure 300 with corresponding functional equations for a CRCNN.
Two neural sub-structures 310, 320 are linked together with this CRCNN. A first internal sub-status st (311-314) of the first neural sub-structure 310 and a second internal sub-status rt (321-324) of the second neural sub-structure 320 are thereby a function according to the following rules of the input quantity ut (301-304), the first internal sub-status st−1 (311-314), the second internal sub-status rt+1 (321-324) and parameter vectors vs, vt, vy:st=NN(vs,st−1, ut),  (8)rt=NN(vr,rt+1,ut)  (9)yt=NN(vy,st,rt)  (10)where NN(.) refers to a mapping rule predefined by the neural network.
Connections between the neurons of the CRCNN are weighted. The weightings are combined in parameter vectors vs, vt, vy.
The CRCNN 300 according to FIG. 3 is a neural network developed over four points in time, t−1, t, t+1, t+2) (see TDRNN, FIG. 5).
Essential features of a neural network developed over a finite number of points in time are described in David E. Rumelhart et al. and above in relation to the known TDRNN (see FIG. 5).
An overview of the basic principles of neural networks and the possible applications of neural networks in the economic field can also be found in H. Rehkugler and H. G. Zimmermann,“Neuronale Netze in der Ökonomie, Grundlagen und finanzwirtschaftliche Anwendungen (Neural networks in economics, Basic principles and financial applications)”, Verlag Franz Vahlen Munich, ISBN 3-8006-1871-0, pages 3-90, 1994.
The known systems and methods in particular have the disadvantage that they only describe a dynamic system to be described with inadequate accuracy and they are therefore only able to forecast future developments of the system with inadequate accuracy.
This is particularly true of dynamic systems, which are subject to a planning influence, i.e. dynamic systems that are not just purely market-driven.